Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Discover] Fix timefield sorting when switching similar index patterns #116145

Merged
merged 14 commits into from
Nov 23, 2021

Conversation

dimaanj
Copy link
Contributor

@dimaanj dimaanj commented Oct 25, 2021

Summary

Fixes #108186
Fixes #117617

Sorting should be set properly after changing index patterns with the same field names, but different timefields. Testing can be done using data provided in the issue.

Checklist

@dimaanj dimaanj added Feature:Discover Discover Application release_note:fix v8.0.0 Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL. v7.16.1 labels Oct 25, 2021
@dimaanj dimaanj self-assigned this Oct 25, 2021
@dimaanj dimaanj marked this pull request as ready for review October 25, 2021 19:41
@dimaanj dimaanj requested a review from a team as a code owner October 25, 2021 19:41
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

@dimaanj
Copy link
Contributor Author

dimaanj commented Oct 30, 2021

@elasticmachine merge upstream

@dimaanj
Copy link
Contributor Author

dimaanj commented Nov 3, 2021

@elasticmachine merge upstream

@elastic elastic deleted a comment from kibanamachine Nov 15, 2021
@dimaanj dimaanj requested a review from kertal November 16, 2021 08:20
@dimaanj dimaanj requested a review from a team as a code owner November 18, 2021 09:07
@@ -290,11 +298,11 @@ export class DataView implements IIndexPattern {
return [...this.fields.getAll().filter((field) => field.scripted)];
}

isTimeBased(): boolean {
isTimeBased(): this is TimeBasedDataView {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ This way we get better TS support. Calling that method now, will cause the compiler to understand, that if that returns true timeFieldName must be a string and can't be undefined anymore.

@dimaanj
Copy link
Contributor Author

dimaanj commented Nov 18, 2021

@elasticmachine merge upstream

Copy link
Contributor

@mattkime mattkime left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, nice improvement! code lgtm!

Copy link
Member

@kertal kertal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thx for all those changes! ❤️ So I did some 🥣 a-la-carte testing

  1. I've installed our ecommerce sample data kibana_sample_data_ecommerce (timeField order_date)
  2. I created a new data view *ecommerce* (timeField: products.created_on)
  3. I've selected kibana_sample_data_ecommerce in Discover, no changes
  4. When I switched from kibana_sample_data_ecommerce (no sort change) to *ecommerce* , the documents were now sorted by products.created_on ... this is how it should behave 👍

Before it was still sorted by order_date

  1. Now I switched to kibana_sample_data_ecommerce,
  2. Sorted by products.product_name.keyword,order_date, so it's a non default sort with the timefield on the second place.
  3. Switched to *ecommerce*
  4. The result was, that the sorting was kept, which I expected to be fine, but since there's a different time column configured, there's no more way to remove the sorting unless changing the URL:
    Discover_-Elastic_und__Discover__Fix_timefield_sorting_when_switching_similar_index_patterns_by_Dmitriynj·Pull_Request__116145·_elastic_kibana
    So I think we need to consider also this case, but I'm not 100% sure how.
    There are 2 ways
  5. We remove the timefield of the previous data field in this case (🚀)
  6. We replace it with the timefield the user has just selected (❤️ )
    Adding another 2 pair of eyes here @majagrubic @timroes

@dimaanj
Copy link
Contributor Author

dimaanj commented Nov 22, 2021

@elasticmachine merge upstream

Copy link
Member

@kertal kertal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added 2 hopefully last comments. thx so much 🙏 for your patience

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
dataViews 526 528 +2

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
discover 312.5KB 312.6KB +140.0B
Unknown metric groups

API count

id before after diff
dataViews 668 671 +3

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @dmitriynj

@kertal kertal self-requested a review November 23, 2021 15:21
Copy link
Member

@kertal kertal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thx so much! 👍
Final test by using this data (one of the examples of the issues linked)

Steps to reproduce:
Create 2 indices, and 2 index patterns: timefield is the time field of my_index and timefield_2 is the time field of my_index_2. In my_index_2 also add timefield to the mapping but it won't contain data:

PUT my_index
{
  "mappings": {
    "properties": {
      "timefield": {
        "type": "date"
      }
    }
  }
}

POST my_index/_bulk
{ "index": {} }
{ "timefield": "2021-08-05" }
{ "index": {} }
{ "timefield": "2021-08-04" }
{ "index": {} }
{ "timefield": "2021-08-03" }

PUT my_index_2
{
  "mappings": {
    "properties": {
      "timefield": {
        "type": "date"
      },
      "timefield_2": {
        "type": "date"
      }
    }
  }
}

POST my_index_2/_bulk
{ "index": {} }
{ "timefield_2": "2021-08-05" }
{ "index": {} }
{ "timefield_2": "2021-08-04" }
{ "index": {} }
{ "timefield_2": "2021-08-03" }

Now in Discover, when switching from my_index to my_index_2, Discover sees the current sorting field (timefield) exists in my_index_2 and doesn’t change it. Since there is no data in this field in my_index_2, it shows up as unsorted.

With this PR
It's fixed 🎂 , switching the to my_index_2 also switches the timeField, so it works

@dimaanj dimaanj added the auto-backport Deprecated - use backport:version if exact versions are needed label Nov 23, 2021
@dimaanj dimaanj merged commit aed58ae into elastic:main Nov 23, 2021
kibanamachine added a commit to kibanamachine/kibana that referenced this pull request Nov 23, 2021
elastic#116145)

* [Discover] fix timefield sorting

* [Discover] resolve review comments

* [Discover] replace primary time field for sorting

* [Discover] improve solution

* Improve typing and time based checking

* Fix missing isTimeBased call

* [Discover] fix linting

* [Discover] replace time fields when switching between time based data views

* [Discover] apply suggestions

Co-authored-by: Kibana Machine <[email protected]>
Co-authored-by: Tim Roes <[email protected]>
@kibanamachine
Copy link
Contributor

💔 Backport failed

Status Branch Result
8.0
7.16 Commit could not be cherrypicked due to conflicts

Successful backport PRs will be merged automatically after passing CI.

To backport manually run:
node scripts/backport --pr 116145

kibanamachine added a commit that referenced this pull request Nov 23, 2021
#116145) (#119492)

* [Discover] fix timefield sorting

* [Discover] resolve review comments

* [Discover] replace primary time field for sorting

* [Discover] improve solution

* Improve typing and time based checking

* Fix missing isTimeBased call

* [Discover] fix linting

* [Discover] replace time fields when switching between time based data views

* [Discover] apply suggestions

Co-authored-by: Kibana Machine <[email protected]>
Co-authored-by: Tim Roes <[email protected]>

Co-authored-by: Dmitry Tomashevich <[email protected]>
Co-authored-by: Tim Roes <[email protected]>
dmlemeshko pushed a commit that referenced this pull request Nov 29, 2021
#116145)

* [Discover] fix timefield sorting

* [Discover] resolve review comments

* [Discover] replace primary time field for sorting

* [Discover] improve solution

* Improve typing and time based checking

* Fix missing isTimeBased call

* [Discover] fix linting

* [Discover] replace time fields when switching between time based data views

* [Discover] apply suggestions

Co-authored-by: Kibana Machine <[email protected]>
Co-authored-by: Tim Roes <[email protected]>
TinLe pushed a commit to TinLe/kibana that referenced this pull request Dec 22, 2021
elastic#116145)

* [Discover] fix timefield sorting

* [Discover] resolve review comments

* [Discover] replace primary time field for sorting

* [Discover] improve solution

* Improve typing and time based checking

* Fix missing isTimeBased call

* [Discover] fix linting

* [Discover] replace time fields when switching between time based data views

* [Discover] apply suggestions

Co-authored-by: Kibana Machine <[email protected]>
Co-authored-by: Tim Roes <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport Deprecated - use backport:version if exact versions are needed Feature:Discover Discover Application release_note:fix Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL. v7.16.1 v8.0.0 v8.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Kibana "Discover" default sorting policy Sorting field in Discover is not updated on index pattern changes
7 participants